Skeletons for parallel image processing: an overview of the SKiPPER project

International audienceThis paper is a general overview of the SKIPPER project, run at Blaise Pascal University between 1996 and 2002. The main goal of the SKIPPER project was to demonstrate the appli- cability of skeleton-based parallel programming techniques to the fast prototyping of reactive vision applications. This project has produced several versions of a full-fledged integrated pa- rallel programming environment (PPE). These PPEs have been used to implement realistic vi- sion applications, such as road following or vehicle tracking for assisted driving, on embedded parallel platforms embarked on semi-autonomous vehicles. All versions of SKIPPER share a common front-end and repertoire of skeletons--presented in previous papers--but differ in the techniques used for implementing skeletons. This paper focuses on these implementation issues, by making a comparative survey, according to a set of four criteria (efficiency, expres- sivity, portability, predictability), of these implementation techniques. It also gives an account of the lessons we have learned, both when dealing with these implementation issues and when using the resulting tools for prototyping vision applications

A Programmable Vision Chip with High Speed Image Processing

International audienceA high speed Analog VLSI Image acquisition and pre-processing system is described in this paper. A 64×64 pixel retina is used to extract the magnitude and direction of spatial gradients from images. So, the sensor implements some low-level image processing in a massively parallel strategy in each pixel of the sensor. Spatial gradients, various convolutions as Sobel filter or Laplacian are described and implemented on the circuit. The retina implements in a massively parallel way, at pixel level, some various treatments based on a four-quadrants multipliers architecture. Each pixel includes a photodiode, an amplifier, two storage capacitors and an analog arithmetic unit. A maximal output frame rate of about 10000 frames per second with only image acquisition and 2000 to 5000 frames per second with image processing is achieved in a 0.35 μm standard CMOS process. The retina provides address-event coded output on three asynchronous buses, one output is dedicated to the gradient and both other to the pixel values. A prototype based on this principle, has been designed. Simulation results from Mentor GraphicsTMsoftware and AustriaMicrosystem Design kit are presented

Hardware-based smart camera for recovering high dynamic range video from multiple exposures

International audienceIn many applications such as video surveillance or defect detection, the perception of information related to a scene is limited in areas with strong contrasts. The high dynamic range (HDR) capture technique can deal with these limitations. The proposed method has the advantage of automatically selecting multiple exposure times to make outputs more visible than fixed exposure ones. A real-time hardware implementation of the HDR technique that shows more details both in dark and bright areas of a scene is an important line of research. For this purpose, we built a dedicated smart camera that performs both capturing and HDR video processing from three exposures. What is new in our work is shown through the following points: HDR video capture through multiple exposure control, HDR memory management, HDR frame generation, and rep- resentation under a hardware context. Our camera achieves a real-time HDR video output at 60 fps at 1.3 mega- pixels and demonstrates the efficiency of our technique through an experimental result. Applications of this HDR smart camera include the movie industry, the mass-consumer market, military, automotive industry, and sur- veillanc

Design, Implementation and Evaluation of Hardware Vision Systems Dedicated to Real-Time Face Recognition

Human face recognition is an active area of research spanning several disciplines such as image processing, pattern recognition, and computer vision. Most researches have concentrated on the algorithms of segmentation, feature extraction, and recognition of human faces, which are generally realized by software implementation on standard computers. However, many applications of human face recognition such as human-computer interfaces, model-based video coding, and security control (Kobayashi, 2001, Yeh & Lee, 1999) need to be high-speed and real-time, for example, passing through customs quickly while ensuring security. For the last years, our laboratory has focused on face processing and obtained interesting results concerning face tracking and recognition by implementing original dedicated hardware systems. Our aim is to implement on embedded systems efficient models of unconstrained face tracking and identity verification in arbitrary scenes. The main goal of these various systems is to provide efficient robustness algorithms that only require moderated computation in order 1) to obtain high success rates of face tracking and identity verification and 2) to cope with the drastic real-time constraints. The goal of this chapter is to describe three different hardware platforms dedicated to face recognition. Each of them has been designed, implemented and evaluated in our laboratory

High Dynamic Range Adaptive Real-time Smart Camera: an overview of the HDR-ARTiST project

International audienceStandard cameras capture only a fraction of the information that is visible to the human visual system. This is specifically true for natural scenes including areas of low and high illumination due to transitions between sunlit and shaded areas. When capturing such a scene, many cameras are unable to store the full Dynamic Range (DR) resulting in low quality video where details are concealed in shadows or washed out by sunlight. The imaging technique that can overcome this problem is called HDR (High Dynamic Range) imaging. This paper describes a complete smart camera built around a standard off-the-shelf LDR (Low Dynamic Range) sensor and a Virtex-6 FPGA board. This smart camera called HDR-ARtiSt (High Dynamic Range Adaptive Real-time Smart camera) is able to produce a real-time HDR live video color stream by recording and combining multiple acquisitions of the same scene while varying the exposure time. This technique appears as one of the most appropriate and cheapest solution to enhance the dynamic range of real-life environments. HDR-ARtiSt embeds real-time multiple captures, HDR processing, data display and transfer of a HDR color video for a full sensor resolution (1280 1024 pixels) at 60 frames per second. The main contributions of this work are: (1) Multiple Exposure Control (MEC) dedicated to the smart image capture with alternating three exposure times that are dynamically evaluated from frame to frame, (2) Multi-streaming Memory Management Unit (MMMU) dedicated to the memory read/write operations of the three parallel video streams, corresponding to the different exposure times, (3) HRD creating by combining the video streams using a specific hardware version of the Devebecs technique, and (4) Global Tone Mapping (GTM) of the HDR scene for display on a standard LCD monitor

HDR-ARtiSt: A 1280x1024-pixel Adaptive Real-time Smart camera for High Dynamic Range video

International audienceStandard cameras capture only a fraction of the information that is visible to the human visual system. This is specifically true for natural scenes including areas of low and high illumination due to transitions between sunlit and shaded areas. When capturing such a scene, many cameras are unable to store the full Dynamic Range (DR) resulting in low quality video where details are concealed in shadows or washed out by sunlight.The imaging technique that can overcome this problem is called HDR (High Dynamic Range) imaging. This paper describes a complete smart camera built around a standard off-the-shelf LDR (Low Dynamic Range) sensor and a Virtex 6 FPGA board. This smart camera called HDR-ARtiSt (High Dynamic Range Adaptive Real-time Smart camera) is able to produce a real-time HDR live video color stream by recording and combining multiple acquisitions of the same scene while varying the exposure time. This technique appears as one of the most appropriate and cheapest solution to enhance the dynamic range of real-life environments. HDR-ARtiSt embeds real-time multiple captures, HDR processing, data display and transfer of a HDR color video for a full sensor resolution (1280 × 1024 pixels) at 60 frames per second. The main contributions of this work are: (1) Multiple Exposure Control (MEC) dedicated to the smart image capture from the sensor with alternating three exposure times that are dynamically evaluated from frame to frame, (2) Multi-streaming Memory Management Unit (MMMU) dedicated to the memory read/write operations of the three parallel video streams

A high speed programmable focal-plane SIMD vision chip. Analog Integrated Circuits and Signal Processing

International audienceA high speed analog VLSI image acquisition and low-level image processing system is presented. The architecture of the chip is based on a dynamically reconfigurable SIMD processor array. The chip features a massively parallel architecture enabling the computation of programmable mask-based image processing in each pixel. Each pixel include a photodiode, an amplifier, two storage capacitors, and an analog arithmetic unit based on a four-quadrant multiplier architecture. A 64 × 64 pixel proof-of-concept chip was fabricated in a 0.35 μm standard CMOS process, with a pixel size of 35 μm × 35 μm. The chip can capture raw images up to 10,000 fps and runs low-level image processing at a framerate of 2,000–5,000 fp

Dark Count rate measurement in Geiger mode and simulation of a photodiode array, with CMOS 0.35 technology and transistor quenching.

International audienceSome decades ago single photon detection used to be the terrain of photomultiplier tube (PMT), thanks to its characteristics of sensitivity and speed. However, PMT has several disadvantages such as low quantum efficiency, overall dimensions, and cost, making them unsuitable for compact design of integrated systems. So, the past decade has seen a dramatic increase in interest in new integrated single-photon detectors called Single-Photon Avalanche Diodes (SPAD) or Geiger-mode APD. SPAD detectors fabricated in a standard CMOS technology feature both single-photon sensitivity, and excellent timing resolution, while guarantying a high integration. SPAD are working in avalanche mode above the breakdown level. When an incident photon is captured, a very fast avalanche is triggered, generating an easily detectable current pulse. In this work, we investigate the design of SPAD detectors using the Austriamicrosystems 0.35 μm CMOS Opto technology. A series of different SPADs has been fabricated and benchmarked in order to evaluate a future integration into a SPAD- based image sensor. The main characteristics of each SPAD operating in Geiger-mode are reported: current voltage, breakdown voltage as a function of temperature. From this first set of results, a detailed study of the Dark Count Rate (DCR) has been conducted. Our results show a dark count rate increase with the size of the photodiodes and the temperature (at T=22.5°C, the DCR of a 10μm-photodiode is 2020 count.s-1 while it is 270 count.s-1 at T=- 40°C for a overvoltage of 800mV). We found that the adjustment of overvoltage is very sensitive and depends on the temperature. The temperature will be adjusted for the subsequent experiments. A mathematical model is presented for reproduce the DCR of a single photodiode. We simulated the noise (DCR) of array of 32x32 photo-detectors. Our results show, of course an increase of DCR of 1024, but especially, the probability of having two pulses simultaneously is 0 (without light). By studying these probabilities of occurrence of the pulses, we think we can reduce the DCR of 50% with a statistical method and reduce the crosstalk of 90%. This study is realized in order to prepare the first digital matrices sensor in Geiger mode

Robust Face Recognition System Based on a Multi-Views Face Database

In this chapter, we describe a new robust face recognition system base on a multi-views face database that derives some 3-D information from a set of face images. We attempt to build an approximately 3-D system for improving the performance of face recognition. Our objective is to provide a basic 3-D system for improving the performance of face recognition. The main goal of this vision system is 1) to minimize the hardware resources, 2) to obtain high success rates of identity verification, and 3) to cope with real-time constraints. Using the multi-views database, we address the problem of face recognition by evaluating the two methods PCA and ICA and comparing their relative performance. We explore the issues of subspace selection, algorithm comparison, and multi-views face recognition performance. In order to make full use of the multi-views property, we also propose a strategy of majority voting among the five views, which can improve the recognition rate. Experimental results show that ICA is a promising method among the many possible face recognition methods, and that the ICA algorithm with majority-voting is currently the best choice for our purposes